<!DOCTYPE html>
<html lang="zh-cn">
    <head><meta charset='utf-8'>
<meta name='viewport' content='width=device-width, initial-scale=1'>
<meta name="baidu-site-verification" content="code-N4HqDA2yr7" />
<meta name="msvalidate.01" content="C30AE7336C0C214E141AC1AAC39A9A05" /><meta name='description' content='线性层的反向传播 对于函数 $ Y = XW $ （ 注：$ X $ 是一个 $ m \times n $ 的矩阵，$ W $ 是一个 $ n \times k $ 的矩阵，$ Y $ 是一个 $ m \times k $ 的矩阵。这里的 $ W'><title>机器学习中常用的一些公式推导</title>

<link rel='canonical' href='/machine-learning/theory/'>

<link rel="stylesheet" href="/scss/style.min.732a6bd9762caf6dce004132442f09bcd64101bf70fd8ab25f9c10d85d58f6b7.css"><meta property='og:title' content='机器学习中常用的一些公式推导'>
<meta property='og:description' content='线性层的反向传播 对于函数 $ Y = XW $ （ 注：$ X $ 是一个 $ m \times n $ 的矩阵，$ W $ 是一个 $ n \times k $ 的矩阵，$ Y $ 是一个 $ m \times k $ 的矩阵。这里的 $ W'>
<meta property='og:url' content='/machine-learning/theory/'>
<meta property='og:site_name' content='倔强的贝吉塔'>
<meta property='og:type' content='article'><meta property='article:section' content='Machine-Learning' /><meta property='article:tag' content='machine learning' /><meta property='article:published_time' content='2023-12-18T21:54:24&#43;08:00'/><meta property='article:modified_time' content='2023-12-18T21:54:24&#43;08:00'/>
<meta name="twitter:title" content="机器学习中常用的一些公式推导">
<meta name="twitter:description" content="线性层的反向传播 对于函数 $ Y = XW $ （ 注：$ X $ 是一个 $ m \times n $ 的矩阵，$ W $ 是一个 $ n \times k $ 的矩阵，$ Y $ 是一个 $ m \times k $ 的矩阵。这里的 $ W">
    <link rel="shortcut icon" href="/img/sorcerer.png" />

<script async src="https://www.googletagmanager.com/gtag/js?id=G-V4H6ZTRWB4"></script>
<script>
var doNotTrack = false;
if (!doNotTrack) {
	window.dataLayer = window.dataLayer || [];
	function gtag(){dataLayer.push(arguments);}
	gtag('js', new Date());
	gtag('config', 'G-V4H6ZTRWB4', { 'anonymize_ip': false });
}
</script>

            
<script async src="https://www.googletagmanager.com/gtag/js?id=G-V4H6ZTRWB4"></script>
<script>
  window.dataLayer = window.dataLayer || [];
  function gtag(){dataLayer.push(arguments);}
  gtag('js', new Date());

  gtag('config', 'G-V4H6ZTRWB4');
</script>

    </head>
    <body class="
    article-page has-toc
">
    <script>
        (function() {
            const colorSchemeKey = 'StackColorScheme';
            if(!localStorage.getItem(colorSchemeKey)){
                localStorage.setItem(colorSchemeKey, "dark");
            }
        })();
    </script><script>
    (function() {
        const colorSchemeKey = 'StackColorScheme';
        const colorSchemeItem = localStorage.getItem(colorSchemeKey);
        const supportDarkMode = window.matchMedia('(prefers-color-scheme: dark)').matches === true;

        if (colorSchemeItem == 'dark' || colorSchemeItem === 'auto' && supportDarkMode) {
            

            document.documentElement.dataset.scheme = 'dark';
        } else {
            document.documentElement.dataset.scheme = 'light';
        }
    })();
</script>
<div class="container main-container flex 
    
        extended
    
">
    
        <div id="article-toolbar">
            <a href="/" class="back-home">
                <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-chevron-left" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
  <path stroke="none" d="M0 0h24v24H0z"/>
  <polyline points="15 6 9 12 15 18" />
</svg>



                <span>返回</span>
            </a>
        </div>
    
<main class="main full-width">
    <article class="main-article">
    <header class="article-header">

    <div class="article-details">
    
    <header class="article-category">
        
            <a href="/categories/machine-learning/" >
                machine learning
            </a>
        
    </header>
    

    <h2 class="article-title">
        <a href="/machine-learning/theory/">机器学习中常用的一些公式推导</a>
    </h2>

    

    
    <footer class="article-time">
        
            <div>
                <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-calendar-time" width="56" height="56" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
  <path stroke="none" d="M0 0h24v24H0z"/>
  <path d="M11.795 21h-6.795a2 2 0 0 1 -2 -2v-12a2 2 0 0 1 2 -2h12a2 2 0 0 1 2 2v4" />
  <circle cx="18" cy="18" r="4" />
  <path d="M15 3v4" />
  <path d="M7 3v4" />
  <path d="M3 11h16" />
  <path d="M18 16.496v1.504l1 1" />
</svg>
                <time class="article-time--published">2023-12-18</time>
            </div>
        

        
            <div>
                <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-clock" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
  <path stroke="none" d="M0 0h24v24H0z"/>
  <circle cx="12" cy="12" r="9" />
  <polyline points="12 7 12 12 15 15" />
</svg>



                <time class="article-time--reading">
                    阅读时长: 3 分钟
                </time>
            </div>
        
    </footer>
    
</div>
</header>

    <section class="article-content">
    
    
    <h2 id="线性层的反向传播">线性层的反向传播</h2>
<p>对于函数 $ Y = XW $ （ 注：$ X $ 是一个 $ m \times n $ 的矩阵，$ W $ 是一个 $ n \times k $ 的矩阵，$ Y $ 是一个 $ m \times k $ 的矩阵。这里的 $ W $ 通常代表模型的权重，而 $ X $ 代表输入数据。）</p>
<p>如何求 $ \frac{\partial Y}{\partial W} $呢，通常我们只关心其一个特定的切片，即 $ \frac{\partial Y_{ij}}{\partial W_{rs}} $。</p>
<p>对于单个元素 $ Y_{ij} $ 而言，其是矩阵 $ X $ 的第 $ i $ 行和矩阵 $ W $ 的第 $ j $ 列的点积。因此：</p>
<p>$$ Y_{ij} = \sum_{p=1}^{n} X_{ip} \cdot W_{pj} $$</p>
<p>当我们对 $ Y_{ij} $ 关于 $ W_{rs} $ 求导时，我们只关心 $ W_{rs} $ 这一项，其他项的导数为0。因此：</p>
<p>$$ \frac{\partial Y_{ij}}{\partial W_{rs}} = \frac{\partial}{\partial W_{rs}} \left( \sum_{p=1}^{n} X_{ip} \cdot W_{pj} \right) $$</p>
<p>$$ \frac{\partial Y_{ij}}{\partial W_{rs}} = X_{ir} \cdot \frac{\partial W_{sj}}{\partial W_{rs}} $$</p>
<p>由于 $ \frac{\partial W_{sj}}{\partial W_{rs}} $ 只有当 $ s = r $ 且 $ j = s $ 时才为1，否则为0，因此：</p>
<p>$$ \frac{\partial Y_{ij}}{\partial W_{rs}} = X_{ir} \cdot \delta_{sj} $$</p>
<p>其中 $ \delta_{sj} $ 是Kronecker delta 函数，当 $ s = j $ 时为1，否则为0</p>
<p>因此，$ \frac{\partial Y}{\partial W} $ 的每个元素 $ \frac{\partial Y_{ij}}{\partial W_{rs}} $ 都是一个 $ m \times n \times k $ 的张量，其中 $ i $ 和 $ j $ 分别索引 $ Y $ 的行和列，而 $ r $ 和 $ s $ 分别索引 $ W $ 的行和列。</p>
<p>然而，通常我们不会将 $ \frac{\partial Y}{\partial W} $ 视为一个张量，而是将其简化为一个矩阵。因为在实际应用中，我们通常对整个矩阵 $ W $ 进行更新，而不是单独更新它的每个元素。因此，我们可以将 $ \frac{\partial Y}{\partial W} $ 的所有元素 $ \frac{\partial Y_{ij}}{\partial W_{rs}} $ 合并成一个 $ m \times k \times n $ 的张量，然后将其简化为一个 $ m \times n $ 的矩阵，即 $ X $。这是因为在矩阵乘法中，$ Y $ 的每个元素 $ Y_{ij} $ 只依赖于 $ X $ 的第 $ i $ 行和 $ W $ 的第 $ j $ 列，所以 $ \frac{\partial Y}{\partial W} $ 实际上是一个 $ m \times n $ 的矩阵，而不是一个 $ m \times n \times k $ 的张量，最终有：</p>
<p>$$ \frac{\partial Y}{\partial W} = X $$</p>
<p>可通过此网站 <a class="link" href="https://www.matrixcalculus.org/"  target="_blank" rel="noopener"
    >https://www.matrixcalculus.org/</a> 验证矩阵求导的正确性。</p>
<p>在神经网络的链式求导过程，我们求 $ Y $ 对 $ W $ 的梯度，通常还会乘以 $ Y $ 本身对应的梯度 $grad_y$，$grad_y$ 是 $ m \times k $ 的矩阵，这时候我们需要对 $ \frac{\partial Y}{\partial W} $ 转置。</p>
<h2 id="softmax函数的反向传播">Softmax函数的反向传播</h2>
<p>Softmax函数的定义是：</p>
<p>$$ p_i = \frac{e^{z_i}}{\sum_j e^{z_j}} $$</p>
<p>对于Softmax函数的导数 $ \frac{\partial p_i}{\partial z_j} $， 可以通过以下方式计算：</p>
<p>$$ \frac{\partial p_i}{\partial z_j} = \frac{\partial}{\partial z_j} \left( \frac{e^{z_i}}{\sum_k e^{z_k}} \right) $$</p>
<p>使用商的求导法则，我们得到：</p>
<p>$$ \frac{\partial p_i}{\partial z_j} = \frac{e^{z_i} \cdot \frac{\partial}{\partial z_j}(\sum_k e^{z_k}) - \frac{\partial}{\partial z_j}(e^{z_i}) \cdot \sum_k e^{z_k}}{(\sum_k e^{z_k})^2} $$</p>
<p>简化后，我们得到：</p>
<p>$$ \frac{\partial p_i}{\partial z_j} = \frac{e^{z_i} \cdot \delta_{ij} - e^{z_i} \cdot e^{z_j}}{(\sum_k e^{z_k})^2} $$</p>
<p>其中 $ \delta_{ij} $ 是Kronecker delta函数，当 $ i = j $ 时为1，否则为0。</p>
<p>进一步简化，我们得到：</p>
<p>$$ \frac{\partial p_i}{\partial z_j} = p_i \cdot (1 - \delta_{ij}) - p_i \cdot p_j $$</p>
<p>最终，我们得到Softmax函数的导数为：</p>
<p>$$ \frac{\partial p_i}{\partial z_j} = \begin{cases} p_i (1 - p_j)   \text{  , if } i = j;
\newline -p_i p_j  \text{ , if } i \neq j \end{cases} $$</p>
<p>Softmax函数的导数是一个矩阵，其中每个元素都是输出概率的函数。</p>
<h2 id="relu函数的反向传播">Relu函数的反向传播</h2>
<p>ReLU（Rectified Linear Unit）函数是深度学习中常用的一种激活函数，其数学表达式为：</p>
<p>$$ f(x) = \max(0, x) $$</p>
<p>其反向传播公式为：</p>
<p>$$  \frac{\partial f(x)}{\partial x} = \begin{cases} 0   \text{  , if } x \leq 0;
\newline 1  \text{ , if } x &gt; 0 \end{cases}$$</p>
<h2 id="mse函数的反向传播">MSE函数的反向传播</h2>
<p>MSE（Mean Squared Error）函数是深度学习中常用的一种损失函数，其公式如下：</p>
<p>$$ MSE = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 $$</p>
<p>其中，$ n $ 是数据点的数量，$ y_i $ 是第 $ i $ 个数据点的真实值，$ \hat{y}_i $ 是第 $ i $ 个数据点的预测值。</p>
<p>在反向传播过程中，MSE 损失函数单个预测值 $ \hat{y}_i $ 的梯度可以表示为：</p>
<p>$$ \frac{\partial MSE}{\partial \hat{y}_i} =  -\frac{2}{n} (y_i - \hat{y}_i) $$</p>

</section>


    <footer class="article-footer">
    
    <section class="article-tags">
        
            <a href="/tags/machine-learning/">machine learning</a>
        
    </section>


    
    <section class="article-copyright">
        <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-copyright" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
  <path stroke="none" d="M0 0h24v24H0z"/>
  <circle cx="12" cy="12" r="9" />
  <path d="M14.5 9a3.5 4 0 1 0 0 6" />
</svg>



        <span>Licensed under CC BY-NC-SA 4.0</span>
    </section>
    

</footer>


    
        <link 
                rel="stylesheet" 
                href="https://cdn.jsdelivr.net/npm/katex@0.15.6/dist/katex.min.css"integrity="sha256-J&#43;iAE0sgH8QSz9hpcDxXIftnj65JEZgNhGcgReTTK9s="crossorigin="anonymous"
            ><script 
                src="https://cdn.jsdelivr.net/npm/katex@0.15.6/dist/katex.min.js"integrity="sha256-InsNdER1b2xUewP&#43;pKCUJpkhiqwHgqiPXDlIk7GzBu4="crossorigin="anonymous"
                defer
                >
            </script><script 
                src="https://cdn.jsdelivr.net/npm/katex@0.15.6/dist/contrib/auto-render.min.js"integrity="sha256-y39Mpg7V3D4lhBX4x6O0bUqTV4pSrfgwEfGKfxkOdgI="crossorigin="anonymous"
                defer
                >
            </script><script>
    window.addEventListener("DOMContentLoaded", () => {
        renderMathInElement(document.querySelector(`.article-content`), {
            delimiters: [
                { left: "$$", right: "$$", display: true },
                { left: "$", right: "$", display: false },
                { left: "\\(", right: "\\)", display: false },
                { left: "\\[", right: "\\]", display: true }
            ]
        });})
</script>
    
</article>

    

    <aside class="related-contents--wrapper">
    
    
        <h2 class="section-title"></h2>
        <div class="related-contents">
            <div class="flex article-list--tile">
                
                    
<article class="">
    <a href="/p/machine-learning/">
        
        

        <div class="article-details">
            <h2 class="article-title">Machine Learning</h2>
        </div>
    </a>
</article>
                
            </div>
        </div>
    
</aside>

     
    
        
    <script src='//unpkg.com/valine@latest/dist/Valine.min.js'></script>

<div id="vcomments"></div>
<style>
    #vcomments .vcards .vcard {
        background-color: var(--card-background);
	    padding: 25px 25px 5px 25px;
	    border-radius: 3px;
	    margin-bottom: 15px;
	    transition: all .3s
    }

    #vcomments .vcards .vh .vquote .vcard {
	    padding: 10px 25px 0px 0px;
    }

    #vcomments .vcards .vcard .vh .vcard {
        border: none;
        box-shadow: none;
    }

    #vcomments .vcards .vcard .vquote .vimg {
        width: 50px;
        height: 50px;
    }

    #vcomments .vcards .vcard .vimg {
        width: 60px;
        height: 60px;
        float: left;
        border-radius: 5px;
        margin-right: 18px;
        border: 0px;
        padding: 0px;
        box-shadow: 0px 10px 20px rgba(0, 0, 0, 0.04), 0px 2px 6px rgba(0, 0, 0, 0.04), 0px 0px 1px rgba(0, 0, 0, 0.04);
    }

    #vcomments .vcards .vcard .vcontent p{
        font-size: 1.6rem;
        color: var(--card-text-color-main);
    }

    #vcomments .vcards .vcard .vcontent{
        padding-top: 0px;
    }
    
    #vcomments .vcount {
        padding: 0px;
        margin-bottom: 5px;
        margin-top: 20px;
        font-weight: 700;
        font-size: 1.4rem;
        color: var(--body-text-color);
        text-transform: uppercase;
    }

    #vcomments .vwrap .cancel-reply-btn {
        position: relative;
        
    }

    #vcomments .vreply-wrapper .vwrap {
        box-shadow: var(--shadow-l2);
        border: 1px solid rgba(143, 143, 143, 0.3);
        width: 95%;
    }

    #vcomments .vquote .vreply-wrapper .vwrap {
        width: 93%;
    }

    #vcomments .vwrap {
        background-color: var(--card-background);
        border: 0px solid #f0f0f0;
        border-radius: 3px;
        margin-bottom: 10px;
        overflow: hidden;
        position: relative;
        padding: 0px 0px 0px 0px;
        font-family: inherit;
        display: flex;
        flex-direction: column;
    }

    #vcomments .vwrap .cancel-reply {
        position: absolute;
        z-index:999;
        float: right;
        top: -20px;
        right: 0px;
    }

    #vcomments .vwrap .vheader.item3{
        order: 0;
    }

    #vcomments .vwrap .edit{
        order: 1;
    }

    #vcomments .vwrap .vinput.vpreview{
        order: 2;
    }

    #vcomments .vwrap .vemojis{
        order: 3;
    }

    #vcomments .vwrap .vrow{
        order: 4;
    }

    #vcomments .vwrap .vmark{
        order: 5;
    }

    #vcomments .vpanel {
        order: 0;
    }

    #vcomments .vload-top.text-center {
        padding-top: 20px;
        order: 1;
    }

    #vcomments .vcount {
        order: 2;
    }

    #vcomments .vcards {
        order: 3;
    }

    #vcomments .vload-bottom.text-center {
        order: 4;
    }

    #vcomments .vempty {
        order: 5;
    }

    #vcomments .vpage.txt-center {
        order: 6;
    }

    #vcomments .vpower.txt-right {
        order: 7;
    }
    
    #vcomments {
        display: flex;
        flex-direction: column;
    }

    #vcomments .vcards .vcard .vhead .vnick {
        color: var(--accent-color);
        position: relative;
        font-size: 1.6rem;
        font-weight: 700;
        margin-right: .875em;
        cursor: pointer;
        text-decoration: none;
        display: inline-block;
    }
    #vcomments .vcards .vcard .vh .vtime {
        color: var(--card-text-color-secondary);
        font-weight: lighter;
        font-size: 1.3rem;
        margin-right: .875em;
    }

    #vcomments .veditor {
        font-size: 1.6rem;
        color: var(--card-text-color-main);
        padding: 20px;
        max-height: 350px;
        resize: none;
    }
    
    #vcomments .vedit {
        border-bottom: 1px solid rgba(143, 143, 143, 0.3);
        background-color: rgba(255, 255, 255, 0.08);
        padding-top: 0px;
    }    

    #vcomments .vrow {
        padding: 20px;
    }

    #vcomments .vedit .vrow {
        padding: 0px 20px 5px 20px;
    }

    #vcomments .vicon.vemoji-btn {
        fill: var(--card-text-color-tertiary);
    }

    #vcomments .vicon.vpreview-btn {
        display:none !important;
    }

    #vcomments .vwrap .vemojis {
        padding: 20px;
    }

    #vcomments .vquote .vmeta .vat {
        right: 30px;
    }

    #vcomments .vmeta .vat {
        background: 0 0;
        cursor: pointer;
        border: 0;
        border-radius: 5px;
        text-transform: uppercase;
        color: rgba(255, 255, 255, 0.5);
        transition: box-shadow .3s ease;
        height: 60px;
        width: 60px;
        display: inline-flex;
        justify-content: center;
        align-items: center;
        position: absolute;
        top: -20px;
        right: 10px;
    }

    #vcomments .vmeta .vat svg {
        stroke-width: 1.33;
    }
    
    #vcomments .vmeta .vat:hover {
        outline: 0;
        box-shadow: var(--shadow-l2);
    }

    #vcomments .vheader {
        display: flex;
    }

    #vcomments .vheader p {
        display: flex;
        flex-direction: column;
        align-items: flex-start;
        flex: 1;
        position: relative;
        margin: 0;
    }

    #vcomments .vheader label {
        position: absolute;
        top: 10px;
        left: 20px;
        color: var(--card-text-color-tertiary);
        margin-top: -4px;
        font-size: 14px;
    }

    #vcomments .vheader input {
         
        color: var(--card-text-color-main);
        border: 0;
        background: 0 0;
        padding: 35px 20px 15px;
        width: 100%;
        line-height: 1;
        transition: border .3s ease;
        font-size: 100%;
    }

    #vcomments .vheader p:not(:last-of-type) {
        border-right: 1px solid rgba(143, 143, 143, 0.3);
    }
    

    #vcomments .vh .vhead {
        padding-top: 5px;
    }

    #vcomments .vh .vquote .vhead {
        padding-top: 0px;
    }

    #vcomments .vbtn {
        color: var(--card-text-color-main);
        font-weight: 700;
        font-size: 1.0em;
        border-radius: 3px;
        border-color: 1px solid #666666;
    }

    #vcomments .vmore.vbtn {
        font-size: 13px;
    }

    #vcomments .vbtn:hover {
        color: #bebebe;
        border-color: #707070;
    }

    #vcomments .vbtn:active {
        color: #bebebe;
        border-color: #707070;
    }

    #vcomments .vcards .vcard .vcontent.expand {
        cursor: auto;
        max-height: none;
        overflow: visible;
    }

    #vcomments .vcards .vcard .vcontent.expand::after {
        display:none !important;
    }

    #vcomments .vcards .vcard .vcontent.expand::before {
        
        display:none !important;
    }

    #vcomments .vsys {
        display:none !important;
    }

    #vcomments .vcards .vcard .vh {
        padding-bottom: 0px;
        border-bottom: 0px dashed #f5f5f5;
    }

    #vcomments .vcards .vcard .vquote {
        padding-top: 5px;
        padding-left: 15px;
        border-left: 0px;
        px solid rgba(187, 187, 187, 0.5);
    }

    #vcomments .vicon.actived {
        fill: var(--card-text-color-main);
    }


</style>

<script>
    
    let placeholder = `发表你的评论\n`

    var valine = new Valine();

    valine.init({
        el: '#vcomments',
        appId: 'KHkYnFUrXOhcAuIDuVXPRFRy-gzGzoHsz',
        appKey: 'VR5IeFKU36DHOY3tzXQo0aBM',
        avatar: 'robohash',
        placeholder: placeholder,
        enableQQ: true,
        lang:'zh-CN',
        requiredFields: [],
        recordIP: false,
        visitor: true
    })

    html = `<p><label>昵称（必填）</label><input name='nick' placeholder='张三' class='vnick vinput' type='text'></p>
            <p><label>邮箱</label><input name='mail' placeholder='joe@example.com' class='vmail vinput' type='email'></p>
            <p><label>网址</label><input name='link' placeholder='https://' class='vlink vinput' type='text'></p>`
            
    document.querySelector(".vheader").innerHTML = html
    document.querySelector(".vpower.txt-right").innerHTML = ""

    if(window.location.hash){
        var checkExist = setInterval(function() {
           if (document.getElementById(window.location.hash.substring(1))) {
              document.getElementById(window.location.hash.substring(1)).scrollIntoView({behavior: "smooth", block: "center", inline: "nearest"})
              clearInterval(checkExist);
           }
        }, 100);
    }

</script>


    

    <footer class="site-footer">
    <section class="copyright">
        &copy; 
        
            2022 - 
        
        2024 倔强的贝吉塔
    </section>
    
    <section class="powerby">
        Built with <a href="https://gohugo.io/" target="_blank" rel="noopener">Hugo</a> <br />
        主题 <b><a href="https://github.com/CaiJimmy/hugo-theme-stack" target="_blank" rel="noopener" data-version="3.7.0">Stack</a></b> 由 <a href="https://jimmycai.com" target="_blank" rel="noopener">Jimmy</a> 设计
    </section>
</footer>


    
<div class="pswp" tabindex="-1" role="dialog" aria-hidden="true">

    
    <div class="pswp__bg"></div>

    
    <div class="pswp__scroll-wrap">

        
        <div class="pswp__container">
            <div class="pswp__item"></div>
            <div class="pswp__item"></div>
            <div class="pswp__item"></div>
        </div>

        
        <div class="pswp__ui pswp__ui--hidden">

            <div class="pswp__top-bar">

                

                <div class="pswp__counter"></div>

                <button class="pswp__button pswp__button--close" title="Close (Esc)"></button>

                <button class="pswp__button pswp__button--share" title="Share"></button>

                <button class="pswp__button pswp__button--fs" title="Toggle fullscreen"></button>

                <button class="pswp__button pswp__button--zoom" title="Zoom in/out"></button>

                
                
                <div class="pswp__preloader">
                    <div class="pswp__preloader__icn">
                        <div class="pswp__preloader__cut">
                            <div class="pswp__preloader__donut"></div>
                        </div>
                    </div>
                </div>
            </div>

            <div class="pswp__share-modal pswp__share-modal--hidden pswp__single-tap">
                <div class="pswp__share-tooltip"></div>
            </div>

            <button class="pswp__button pswp__button--arrow--left" title="Previous (arrow left)">
            </button>

            <button class="pswp__button pswp__button--arrow--right" title="Next (arrow right)">
            </button>

            <div class="pswp__caption">
                <div class="pswp__caption__center"></div>
            </div>

        </div>

    </div>

</div><script 
                src="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe.min.js"integrity="sha256-ePwmChbbvXbsO02lbM3HoHbSHTHFAeChekF1xKJdleo="crossorigin="anonymous"
                defer
                >
            </script><script 
                src="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe-ui-default.min.js"integrity="sha256-UKkzOn/w1mBxRmLLGrSeyB4e1xbrp4xylgAWb3M42pU="crossorigin="anonymous"
                defer
                >
            </script><link 
                rel="stylesheet" 
                href="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/default-skin/default-skin.min.css"crossorigin="anonymous"
            ><link 
                rel="stylesheet" 
                href="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe.min.css"crossorigin="anonymous"
            >

            </main>
    
        <aside class="sidebar right-sidebar sticky">
            <section class="widget archives">
                <div class="widget-icon">
                    <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-hash" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
  <path stroke="none" d="M0 0h24v24H0z"/>
  <line x1="5" y1="9" x2="19" y2="9" />
  <line x1="5" y1="15" x2="19" y2="15" />
  <line x1="11" y1="4" x2="7" y2="20" />
  <line x1="17" y1="4" x2="13" y2="20" />
</svg>



                </div>
                <h2 class="widget-title section-title">目录</h2>
                
                <div class="widget--toc">
                    <nav id="TableOfContents">
  <ol>
    <li><a href="#线性层的反向传播">线性层的反向传播</a></li>
    <li><a href="#softmax函数的反向传播">Softmax函数的反向传播</a></li>
    <li><a href="#relu函数的反向传播">Relu函数的反向传播</a></li>
    <li><a href="#mse函数的反向传播">MSE函数的反向传播</a></li>
  </ol>
</nav>
                </div>
            </section>
        </aside>
    

        </div>
        <script 
                src="https://cdn.jsdelivr.net/npm/node-vibrant@3.1.6/dist/vibrant.min.js"integrity="sha256-awcR2jno4kI5X0zL8ex0vi2z&#43;KMkF24hUW8WePSA9HM="crossorigin="anonymous"
                
                >
            </script><script type="text/javascript" src="/ts/main.js" defer></script>
<script>
    (function () {
        const customFont = document.createElement('link');
        customFont.href = "https://fonts.googleapis.com/css2?family=Lato:wght@300;400;700&display=swap";

        customFont.type = "text/css";
        customFont.rel = "stylesheet";

        document.head.appendChild(customFont);
    }());
</script>

    </body>
</html>
